Papers with monolingual applications
On the Limitations of Language-targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning (2026.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in large language model pruning have shown high predictive performance in post-training settings. |
| Approach: | They conduct an empirical study on the performance and internal representation changes associated with pruning multilingual models for monolingual applications. |
| Outcome: | The proposed pruning methods retain perplexity and yield high signal-to-noise ratios, but not consistently improve downstream tasks. |
The Impact of Positional Encodings on Multilingual Compression (2021.emnlp-main)
Copied to clipboard
| Challenge: | Several modifications have been proposed to improve monolingual language models, but none of them result in better multilingual models. |
| Approach: | They propose to add positional encodings to token embeddings to preserve word-order information in a non-autoregressive setting. |
| Outcome: | The proposed modifications tend to improve monolingual models, but none improve multilingual models. |